Content-based video indexing for sports applications using integrated multi-modal approach

This thesis presents a research work based on an integrated multi-modal approach for sports video indexing and retrieval. By combining specific features extractable from multiple (audio-visual) modalities, generic structure and specific events can be detected and classified. During browsing and retrieval, users will benefit from the integration of high-level semantic and some descriptive mid-level features such as whistle and close-up view of player(s). The main objective is to contribute to the three major components of sports video indexing systems. The first component is a set of powerful techniques to extract audio-visual features and semantic contents automatically. The main purposes are to reduce manual annotations and to summarize the lengthy contents into a compact, meaningful and more enjoyable presentation. The second component is an expressive and flexible indexing technique that supports gradual index construction. Indexing scheme is essential to determine the methods by which users can access a video database. The third and last component is a query language that can generate dynamic video summaries for smart browsing and support user-oriented retrievals

An Adaptive and Extensible Web-based Interface System for Interactive Video Contents Browsing

With the growing popularity of mobile devices (including phones and portable media players) and coverage of Internet access, we tend to develop the need of consuming video content on the move. Some technologies already allow end-users to watch TV and listen to news podcasts or download music videos on their devices. However, such services are restricted to a provider's selection of preformatted and linear content streams. Hence, we propose a web-based interface system that supports interactive contents navigation, making it possible for end-users to "surf" on video content like they are used to on the Web. This system is extensible to any specific domain of video contents, any webenabled platform, and to any browsing scheme. In this paper, we will explain the architecture and design of this system, propose an application for soccer videos and present the results of its user evaluation

PhD Thesis: "Content-based Video Indexing for Sports Applications using Multi-modal approach"

Triggered by technology innovations, there has been a huge increase in the utilization of video, as one of the most preferred types of media due to its content richness, for many significant applications. To sustain an ongoing rapid growth of video information, there is an emerging demand for a sophisticated content-based video indexing system. However, current video indexing solutions are still immature and lack of any standard. One solution, namely annotation-based indexing, allows video retrieval using textual annotations. However, the major limitations are the restrictions of pre-defined keywords that can be used and the expensive manual work on annotating video. Another solution called feature-based indexing allows video search by low-level features comparison such as query by a sample image. Even though this approach can use automatically extracted features, users would not be able to retrieve video intuitively, based on high-level concepts. This predicament is caused by the so-called semantic gap which highlights the fact that users recall video contents in a high-level abstraction while video is generally stored as an arbitrary sequence of audio-visual tracks. To bridge the semantic gap, this thesis will demonstrate the use of domain-specific approach which aims to utilize domain knowledge in facilitating the extraction of high-level concepts directly from the audiovisual features. The main idea behind domain-specific approach is the use of domain knowledge to guide the integration of features from multi-modal tracks. For example, to extract goal segments from soccer and basketball video, slow motion replay scenes (visual) and excitement (audio) should be detected as they are played during most goal segments. Domain-specific indexing also exploits specific browsing and querying methods which are driven by specific users/applications’ requirements. Sports video is selected as the primary domain due to its content richness and popularity. Moreover, broadcasted sports videos generally span for hours with many redundant activities and the key segments could make up only 30% to 60% of the entire data depending on the progress of the match. This thesis presents a research work based on an integrated multi-modal approach for sports video indexing and retrieval. By combining specific features extractable from multiple (audio-visual) modalities, generic structure and specific events can be detected and classified. During browsing and retrieval, users will benefit from the integration of high-level semantic and some descriptive mid-level features such as whistle and close-up view of player(s). The main objective is to contribute to the three major components of sports video indexing systems. The first component is a set of powerful techniques to extract audio-visual features and semantic contents automatically. The main purposes are to reduce manual annotations and to summarize the lengthy contents into a compact, meaningful and more enjoyable presentation. The second component is an expressive and flexible indexing technique that supports gradual index construction. Indexing scheme is essential to determine the methods by which users can access a video database. The third and last component is a query language that can generate dynamic video summaries for smart browsing and support user-oriented retrievals

Using viewer’s facial expression and heart rate for sports video highlights detection

Viewer interests, evoked by video content, can potentially identify the highlights of the video. This paper explores the use of facial expressions (FE) and heart rate (HR) of viewers captured using camera and non-strapped sensor for identifying interesting video segments. The data from ten subjects with three videos showed that these signals are viewer dependent and not synchronized with the video contents. To address this issue, new algorithms are proposed to effectively combine FE and HR signals for identifying the time when viewer interest is potentially high. The results show that, compared with subjective annotation and match report highlights, ‘non-neutral’ FE and ‘relatively higher and faster’ HR is able to capture 60%-80% of goal, foul, and shot-on-goal soccer video events. FE is found to be more indicative than HR of viewer’s interests, but the fusion of these two modalities outperforms each of them

Web search engine multimedia functionality

Web search engines are beginning to offer access to multimedia searching, including audio, video and image searching. In this paper we report findings from a study examining the state of multimedia search functionality on major general and specialized Web search engines. We investigated 102 Web search engines to examine: (1) how many Web search engines offer multimedia searching, (2) the type of multimedia search functionality and methods offered, such as "query by example", and (3) the supports for personalization or customization which are accessible as advanced search. Findings include: (1) few major Web search engines offer multimedia searching and (2) multimedia Web search functionality is generally limited. Our findings show that despite the increasing level of interest in multimedia Web search, those few Web search engines offering multimedia Web search, provide limited multimedia search functionality. Keywords are still the only means of multimedia retrieval, while other methods such as "query by example" are offered by less than 1% of Web search engines examined

Exploring the opportunities and challenges of using mobile sensing for gamification and achievements

Gamified services delivered on smart phones, such as Foursquare, are able to utilise the sensors on the phone to capture user contexts as a means of triggering game elements. This paper identifies and discusses opportunities \ud and challenges that exist when using mobile sensors as input for game elements. We present initial findings from a field study of a gamified mobile application made to support the university orientation event for new students\ud using game achievements. The study showed that overall the use of context was well received by participants when compared to game elements that required no context to complete. It was also found that using context could help validate that an activity was completed however there were still technical challenges when using sensors that led to exploits in the game elements, or cheating

Published by Springer-Verlag 2005 Multi-level Semantic Analysis for Sports Video

Abstract. There has been a huge increase in the utilization of video as one of the most preferred type of media due to its content richness for many significant applications including sports. To sustain an ongoing rapid growth of sports video, there is an emerging demand for a sophisticated content-based indexing system. Users recall video contents in a high-level abstraction while video is generally stored as an arbitrary sequence of audio-visual tracks. To bridge this gap, this paper will demonstrate the use of domain knowledge and characteristics to design the extraction of high-level concepts directly from audio-visual features. In particular, we propose a multi-level semantic analysis framework to optimize the sharing of domain characteristics.

Molly Lamb Bobak : Une rétrospective

Stressing the artist's lust for life, Richmond provides an overview of the development of Bobak's career spanning 50 years. Focusing on the artist's activities between 1942 and 1945, Foss examines her choice of subject matter and the circumstances surrounding her appointment as Canada's first female war artist. Biographical notes. Bibl. 7 p

Tracking and video surveillance activity analysis

The explosion in the number of cameras surveilling the environment in recent years is generating a need for systems capable of analysing video streams for important events. This paper outlines a system for detecting noteworthy behaviours (from a security or surveillance perspective) which does not involve the enumeration of the event sequences of all possible activities of interest. Instead the focus is on calculating a measure of the abnormality of the action taking place. This raises the need for a low complexity tracking algorithm robust to the noise artefacts present in video surveillance systems. The tracking technique described herein achieves this goal by using a ``future history'' buffer of images and so delaying the classification and tracking of objects by the time quantum which is the buffer size. This allows disambiguation of noise blobs and facilitates classification in the case of occlusions and disappearance of people due to lighting, failures in the background model etc